[BUG FIX]: hf_hub_download crashes when stderr lacks a real file descriptor#4065
Open
tobocop2 wants to merge 5 commits intohuggingface:mainfrom
Open
[BUG FIX]: hf_hub_download crashes when stderr lacks a real file descriptor#4065tobocop2 wants to merge 5 commits intohuggingface:mainfrom
tobocop2 wants to merge 5 commits intohuggingface:mainfrom
Conversation
tobocop2
force-pushed
the
fix/tqdm-bad-fd-safety
branch
from
f08d2e6 to
6eff9fb
Compare
This was referenced Apr 7, 2026
tobocop2
changed the title
_create_progress_bar crashes with ValueError when stderr.fileno() returns -1 (Textual, Jupyter, pytest) or raises OSError. The crash is in tqdm's multiprocessing lock init, not the download itself. Add _SafeTqdm fallback that uses a threading lock instead, and wrap both cls() calls in try/except (OSError, ValueError).
tobocop2
force-pushed
the
fix/tqdm-bad-fd-safety
branch
from
6f4b2a4 to
43399d3
Compare
tobocop2
changed the title
tobocop2
changed the title
The custom-class branch of _create_progress_bar was falling back to _SafeTqdm(**kwargs) without passing disable=True, so after a crash in the caller's class the fallback rendered a visible tqdm bar into the caller's output stream. For TUIs that had opted into custom progress via tqdm_class, this meant ANSI escapes leaking into the app. Pass disable=True on the fallback, matching the HF-subclass branch which already passes disable=disable. Add a regression test covering the crash-in-__init__ path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
hf_hub_downloadcrashes withValueError: bad value(s) in fds_to_keepin any environment wheresys.stderris not backed by a real file descriptor.The download itself is fine. Only tqdm's progress bar initialization fails, but because
_create_progress_barhas no safety net, it takes down the entire download.Affected environments
Any runtime where
sys.stderris not backed by a real file descriptor:Root cause
_create_progress_barcallscls(disable=disable, name=name, **kwargs)with no exception handling. When stderr'sfileno()returns-1, tqdm's__init__tries to create a multiprocessing lock, which spawns a resource tracker subprocess viafork_exec(), which rejects the invalid fd.Crash Demo
Fix
Add a
_SafeTqdmsubclass that uses athreading.RLockinstead of a multiprocessing lock, and wrap bothcls()calls in_create_progress_barwithtry/except (OSError, ValueError)to fall back to_SafeTqdm.Fixes #4066
Note
Medium Risk
Changes progress bar construction and fallback behavior across download paths; while scoped to error handling, it affects user-visible progress output and warning emission in diverse runtimes (Jupyter/TUIs/pytest).
Overview
Prevents
hf_hub_download/snapshot_downloadfrom crashing whentqdmcannot initialize its multiprocessing lock (e.g.sys.stderr.fileno()is invalid) by wrapping_create_progress_barconstruction intry/except (OSError, ValueError).Introduces
_SafeTqdm(threadingRLock-based) as a fallback: HF’s default bar falls back with a warning and continues without progress, while customtqdm_classfallbacks are forceddisable=Trueto avoid corrupting caller output (e.g. TUIs). Adds regression tests covering badfileno()and exploding custom tqdm initialization.Reviewed by Cursor Bugbot for commit cb1cb66. Bugbot is set up for automated code reviews on this repo. Configure here.